Device and method for operating cameras and light sources

ABSTRACT

Provided herein is a device and method for improved operating of cameras and light sources of a mobile automation apparatus. Light sources are operated to periodically provide illumination light for a camera operating according to a given exposure time and frequency, with a pulse duration having a respective frequency that is an integer multiple of the camera frequency, and higher than a threshold frequency where successive activations of the light sources are imperceptible. Furthermore, a light source paired with a camera is located at a distance from the paired camera that illuminates an object imaged by the paired camera, and where parasitic reflections from the paired light source are not reflected into the paired camera.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is related to U.S. Provisional Application No.62/492,670 entitled “Product Status Detection System,” filed on May 1,2017, by Perrella et al., which is incorporated herein by reference inits entirety.

BACKGROUND OF THE INVENTION

Mobile automation apparatuses are increasingly being used in retailenvironments to perform inventory control. For example, such mobileautomation apparatuses are generally equipped with various sensors toperform such tasks, as well as to autonomously navigate paths within theretail environment. During such autonomous navigation, the mobileautomation apparatuses generally perform ancillary tasks that includedata capture using cameras, for example capture of images of bar codes,printed price labels and the like, which can be located behind at leastpartially reflective, generally transparent surfaces. Due to themovement of the mobile automation apparatuses, the cameras are generallyconfigured to operate over very short exposure times. Light sources thatilluminate objects for which images are being acquired must generally bevery bright. As the mobile automation apparatuses are powered from abattery, the light sources should be operated in a manner whichminimizes the power draw on the battery. If, however, the light sourcesare operated only when the cameras are acquiring images, the pulseduration and frequency of the light sources may be such that stroberelated discomfort may be induced in certain sensitive observers.Furthermore, as the mobile automation apparatuses generally include twoare more cameras, parasitic reflections from the light sources caninterfere with the image acquisition.

Accordingly, there is a need for a device and method for improvedoperating of cameras and light sources of a mobile automation apparatusin the above environment.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The accompanying figures, where like reference numerals refer toidentical or functionally similar elements throughout the separateviews, together with the detailed description below, are incorporated inand form part of the specification, and serve to further illustrateembodiments of concepts that include the claimed invention, and explainvarious principles and advantages of those embodiments.

FIG. 1 is a block diagram of a mobile automation system in accordancewith some embodiments.

FIG. 2 is a side view of a mobile automation apparatus in accordancewith some embodiments.

FIG. 3 is a block diagram of the mobile automation apparatus of FIG. 2in accordance with some embodiments.

FIG. 4 is a block diagram of a control application in accordance withsome embodiments.

FIG. 5 is a flowchart of a method of operating a camera and at least onelight source to prevent seizures in accordance with some embodiments.

FIG. 6 is a timing diagram for implementing the method of FIG. 5 inaccordance with some embodiments.

FIG. 7 is a flowchart of a method of operating cameras and light sourcesto prevent parasitic reflections from interfering with image acquisitionin accordance with some embodiments.

FIG. 8 depicts the fields of view of the cameras of the apparatus ofFIG. 2 in accordance with some embodiments.

FIG. 9 depicts a light source producing parasitic reflections in acamera of the apparatus of FIG. 2 in accordance with some embodiments.

FIG. 10 depicts a light source arranged to provide illumination for acamera of the apparatus of FIG. 2, without parasitic reflections, inaccordance with some embodiments.

FIG. 11 depicts geometry of the cameras and the light sources of theapparatus of FIG. 2, as well as a channel arrangement, in accordancewith some embodiments

FIG. 12 is a timing diagram for implementing the method of FIG. 7 inaccordance with some embodiments.

Skilled artisans will appreciate that elements in the figures areillustrated for simplicity and clarity and have not necessarily beendrawn to scale. For example, the dimensions of some of the elements inthe figures may be exaggerated relative to other elements to help toimprove understanding of embodiments of the present invention.

The apparatus and method components have been represented whereappropriate by conventional symbols in the drawings, showing only thosespecific details that are pertinent to understanding the embodiments ofthe present invention so as not to obscure the disclosure with detailsthat will be readily apparent to those of ordinary skill in the arthaving the benefit of the description herein.

DETAILED DESCRIPTION OF THE INVENTION

An aspect of the present specification provides a device comprising: acamera, configured to acquire images periodically according to a givenexposure time, and at a first given frequency; and at least one lightsource configured to periodically provide illumination light for thecamera, the illumination light having a pulse duration that correspondsto the given exposure time and having a second given frequency that isan integer multiple of the first given frequency, wherein the secondgiven frequency is higher than a threshold frequency where successiveactivations of the at least one light source are imperceptible.

In some embodiments, each of the given exposure time and the pulseduration are about 0.5 ms.

In some embodiments, the first given frequency is about 5 Hz and thesecond given frequency is about 100 Hz.

In some embodiments, the threshold frequency is greater than about 70Hz.

In some embodiments, the at least one light source comprises at leastone light emitting diode.

In some embodiments, the second given frequency is higher than thethreshold frequency where the successive activations of the at least onelight source are imperceptible to a human vision system.

In some embodiments, the at least one light source and the camera arepowered by a battery.

In some embodiments, the given exposure time is selected such that theimages acquired by the camera are not blurry when the device is movingat a given speed.

Another aspect of the present specification provides a methodcomprising: at a device comprising controller, a camera and at least onelight source, controlling, using the controller, the camera to acquireimages periodically according to a given exposure time, and at a firstgiven frequency; and controlling, using the controller, the at least onelight source to periodically provide illumination light for the camera,the illumination light having a pulse duration that corresponds to thegiven exposure time and having a second given frequency that is aninteger multiple of the first given frequency, wherein the second givenfrequency is higher than a threshold frequency where successiveactivations of the at least one light source are imperceptible. In someembodiments, the second given frequency is higher than the thresholdfrequency where the successive activations of the at least one lightsource are imperceptible to a human vision system.

A further aspect of the specification provides a non-transitorycomputer-readable medium storing a computer program, wherein executionof the computer program is for: at a device comprising controller, acamera and at least one light source, controlling, using the controller,the camera to acquire images periodically according to a given exposuretime, and at a first given frequency; and controlling, using thecontroller, the at least one light source to periodically provideillumination light for the camera, the illumination light having a pulseduration that corresponds to the given exposure time and having a secondgiven frequency that is an integer multiple of the first givenfrequency, wherein the second given frequency is higher than a thresholdfrequency where successive activations of the at least one light sourceare imperceptible. In some embodiments, the second given frequency ishigher than the threshold frequency where the successive activations ofthe at least one light source are imperceptible to a human vision system

Yet a further aspect of the specification provides a device comprising:a support structure; a plurality of cameras spaced along the supportstructure, wherein adjacent cameras have partially overlapping fields ofview directed away from the support structure; and, a plurality of lightsources comprising at least one light source paired with each of theplurality of cameras, wherein a paired light source is located at adistance from a paired camera that illuminates an object imaged by thepaired camera, and where parasitic reflections from the paired lightsource are not reflected into the paired camera, and wherein the pairedlight source and the paired camera are operated at a different time fromother paired light sources and other paired cameras located where theparasitic reflections from the paired light source are reflected intothe other paired cameras.

In some embodiments, the paired light source and the paired camera areoperated at a same time as associated paired light sources andassociated paired cameras, the paired camera and the associated pairedcameras each located where respective parasitic reflections from thepaired light source and the associated paired light sources are notreflected into the paired camera and the associated paired cameras.

In some embodiments, the plurality of cameras is operated in a sequencewith associated paired light sources.

In some embodiments, the device further comprises a mobile automationapparatus, the support structure disposed on the mobile automationapparatus, the plurality of cameras and the plurality of light sourcesspaced along the support structure.

In some embodiments, the device further comprises a mobile automationapparatus, the support structure comprising a mast disposed on themobile automation apparatus, the plurality of cameras and the pluralityof light sources vertically spaced along the mast.

In some embodiments, at least one camera is located between the pairedlight source and the paired camera along the support structure.

In some embodiments, each of the plurality of light sources is pitched,with respect to the support structure, in a direction of an associatedpaired camera.

In some embodiments, a given camera, of the plurality of cameras ispaired with at least a first subset of the plurality of light sourceslocated on one side of the given camera, and a second subset of theplurality of light sources located on an opposite side of the givencamera.

In some embodiments, associated subsets of the plurality of lightsources are located in rows along the support structure, perpendicularto a line between the plurality of cameras, and wherein at least some ofthe rows are operated at a same time to provide illumination light forat least two of the plurality of cameras.

Yet a further aspect of the specification provides a method comprising:at a device comprising: a controller; a support structure; a pluralityof cameras spaced along the support structure, wherein adjacent camerashave partially overlapping fields of view directed away from the supportstructure; and, a plurality of light sources comprising at least onelight source paired with each of the plurality of cameras, wherein apaired light source is located at a distance from a paired camera thatilluminates an object imaged by the paired camera, and where parasiticreflections from the paired light source are not reflected into thepaired camera, controlling, using the controller, the paired lightsource and the paired camera to operate at a different time from otherpaired light sources and other paired cameras located where theparasitic reflections from the paired light source are reflected intothe other paired cameras; and controlling, using the controller, thepaired light source and the paired camera to operate at a same time asassociated paired light sources and associated paired cameras, thepaired camera and the associated paired cameras each located whererespective parasitic reflections from the paired light source and theassociated paired light sources are not reflected into the paired cameraand the associated paired cameras.

A further aspect of the specification provides a non-transitorycomputer-readable medium storing a computer program, wherein executionof the computer program is for: at a device comprising: a controller; asupport structure; a plurality of cameras spaced along the supportstructure, wherein adjacent cameras have partially overlapping fields ofview directed away from the support structure; and, a plurality of lightsources comprising at least one light source paired with each of theplurality of cameras, wherein a paired light source is located at adistance from a paired camera that illuminates an object imaged by thepaired camera, and where parasitic reflections from the paired lightsource are not reflected into the paired camera, controlling, using thecontroller, the paired light source and the paired camera to operate ata different time from other paired light sources and other pairedcameras located where the parasitic reflections from the paired lightsource are reflected into the other paired cameras; and controlling,using the controller, the paired light source and the paired camera tooperate at a same time as associated paired light sources and associatedpaired cameras, the paired camera and the associated paired cameras eachlocated where respective parasitic reflections from the paired lightsource and the associated paired light sources are not reflected intothe paired camera and the associated paired cameras.

FIG. 1 depicts a mobile automation system 100 in accordance with theteachings of this disclosure. The system 100 includes a server 101 incommunication with at least one mobile automation apparatus 103 (alsoreferred to herein simply as the apparatus 103) and at least one mobiledevice 105 via communication links 107, illustrated in the presentexample as including wireless links. The system 100 is deployed, in theillustrated example, in a retail environment including a plurality ofmodules 110 of shelves each supporting a plurality of products 112. Morespecifically, the apparatus 103 is deployed within the retailenvironment, and at least periodically communicates with the server 101(via the link 107) as it navigates the length of at least a portion ofthe modules 110. The apparatus 103 is equipped with a plurality of datacapture sensors, such as image sensors (e.g. one or more digitalcameras) and depth sensors (e.g. one or more lidar sensors), and isfurther configured to employ the sensors to capture shelf data. In thepresent example, the apparatus 103 is configured to capture a series ofdigital images of the modules 110, as well as a series of depthmeasurements, each describing the distance and direction between theapparatus 103 and a point associated with the module 110, such as theshelf module 110 itself or products disposed thereon.

The server 101 includes a controller 120, interconnected with anon-transitory computer readable storage medium, such as a memory 122.The memory 122 includes any suitable combination of volatile (e.g.Random Access Memory or RAM) and non-volatile memory (e.g. read onlymemory or ROM, Electrically Erasable Programmable Read Only Memory orEEPROM, flash memory). In general, the controller 120 and the memory 122each comprise one or more integrated circuits. The controller 120, forexample, includes a special purpose controller having one or more ofcentral processing units (CPUs) and graphics processing units (GPUs)and/or one or more of field-programmable gate arrays (FPGAs) and/orapplication-specific integrated circuits (ASICs) configured to implementthe specific functionality of the server 101. In an embodiment, thecontroller 120, further includes one or more central processing units(CPUs) and/or graphics processing units (GPUs). In an embodiment, aspecially designed integrated circuit, such as a Field Programmable GateArray (FPGA), is designed to perform specific functionality of theserver 101, either alternatively or in addition to the controller 120and the memory 122. As those of skill in the art will realize, themobile automation apparatus 103 also includes one or more controllers orprocessors and/or FPGAs, in communication with the controller 120,specifically configured to control navigational and/or data captureaspects of the mobile automation apparatus 103.

The server 101 also includes a communications interface 124interconnected with the controller 120. The communications interface 124includes any suitable hardware (e.g. transmitters, receivers, networkinterface controllers and the like) allowing the server 101 tocommunicate with other computing devices—particularly the apparatus 103and the mobile device 105—via the links 107. The links 107 may be directlinks, or links that traverse one or more networks, including both localand wide-area networks. The specific components of the communicationsinterface 124 are selected based on the type of network or other linksthat the server 101 is required to communicate over. In the presentexample, a wireless local-area network is implemented within the retailenvironment via the deployment of one or more wireless access points.The links 107 therefore include both wireless links between theapparatus 103 and the mobile device 105 and the above-mentioned accesspoints, and a wired link (e.g. an Ethernet-based link) between theserver 101 and the access point.

The memory 122 stores a plurality of applications, each including aplurality of computer readable instructions executable by the controller120. The execution of the above-mentioned instructions by the controller120 configures the server 101 to perform various actions discussedherein. The applications stored in the memory 122 include a controlapplication 128, which may also be implemented as a suite of logicallydistinct applications. In general, via execution of the controlapplication 128 or subcomponents thereof, the controller 120 isconfigured to implement various functionality. The controller 120, asconfigured via the execution of the control application 128, is alsoreferred to herein as the controller 120. As will now be apparent, someor all of the functionality implemented by the controller 120 describedbelow may also be performed by preconfigured special purpose hardwareelements (e.g. one or more ASICs) rather than by execution of thecontrol application 128 by the controller 120.

In general, the controller 120 is configured to at least periodicallycommunicate with the mobile automation apparatus 103, which autonomouslynavigates the environment and captures data, to obtain the captured datavia the communications interface 124 and store the captured data in arepository 132 of the memory 122. The server 101 is further configuredto perform various post-processing operations on the captured data, andto detect the status of the products 112 on the modules 110. Whencertain status indicators are detected, the server 101 is alsoconfigured to transmit status notifications to the mobile device 105.

For example, in some embodiments, the server 101 is configured via theexecution of the control application 128 by the controller 120, toprocess image and depth data captured by the apparatus 103 to identifyportions of the captured data depicting a back of a module 110, and todetect gaps between the products 112 based on those identified portions.In some embodiments navigation of the mobile automation apparatus isfully autonomous, while in other embodiments the server 101 facilitatesnavigation of the mobile automation apparatus 103 by providing a mapand/or paths and/or path segments and/or navigation data and/ornavigation instructions to the apparatus 103 to help the apparatus 103navigate among the modules 110.

Attention is next directed to FIG. 2 and FIG. 3 which respectivelydepict: a schematic side perspective view of the apparatus 103, showingparticular sensors of the apparatus 103; and a schematic block diagramof the apparatus 103.

With reference to FIG. 2, the apparatus 103 is being operated adjacent amodule 110 having products 112 thereupon. Further, at least one label201 is attached to a shelf of the module 110, and at least one product112 is behind a door 203 made from glass, and the like. While only onelabel 201 is depicted it is appreciated a plurality of labels 201 isgenerally attached to the shelves of the module 110. The apparatus 103generally comprises: a base 210 configured to move on wheels 212 (e.g.on a floor 213 of the environment), and the like; and a mast 214 and/orsupport structure extending in an upward direction (e.g. vertically)from the base 210. However, in other embodiments, the base 210 moves ondevices other than wheels, for example casters, tracks and the like. Insome embodiments, the base 210 and the mast 214 form a housing, howeverother types of housings, bases and/or support structures will occur topersons of skill in the art and are within the scope of presentembodiments.

The apparatus 103 further includes a plurality of cameras 218-1, 218-2,218-3, 218-4, 218-5, 218-6, 218-7, 218-8 (referred to hereafter,collectively, as the cameras 218 and, generically, as a camera 218), thecameras 218 spaced along the mast 214 and/or support structure, and inparticular vertically spaced along the mast 214 and/or supportstructure, wherein adjacent cameras have partially overlapping fields ofview directed away from the mast 214 and/or support structure. Putanother way, the cameras 218 are directed away from the mast 214 and/orsupport structure that is disposed on the base 212, such that thecenterline of the field of view of each camera 218 is perpendicular tothe mast 214. The apparatus 103 further includes a plurality of lightsources 219-1, 219-2, 219-3, 219-4, 219-5, 219-6, 219-7, 219-8, 219-9,219-10 (referred to hereafter, collectively, as the light sources 219and, generically, as a light source 219), at least one light source 219paired with each of the plurality of cameras 218. When one of thecameras 218 is operated, the paired light source 219 is operated toilluminate a field of view of the camera 218, as described in furtherdetail below.

While not depicted, the base 210 and/or the mast 214 are provisionedwith other various navigation sensors for navigating in the environmentin which the modules 110 are located and include various furtherobstacle detection sensors for detecting and avoiding obstacles.

As depicted the apparatus 103 is navigated along the module 110 (e.g.into the page or out of the page with reference to FIG. 2), and includeseight cameras 218 spaced along the mast 214, and ten light sources 219,the cameras 218 and light sources 219 operated to acquire images of theproducts 112 and/or labels 201 for use in one or of stock detection, lowstock detection, price label verification, plug detection, planogramcompliance and the like. For example, the products 112 and/or the labels201 have barcodes, and the like, to be imaged and decoded. As those ofskill in the art will realize, other numbers of cameras and lightsources are within the scope of present embodiments.

Due to the speed of the apparatus 103 as it moves relative to the module110, each of the cameras 218 are generally configured to acquire imagesperiodically according to a given exposure time, and at a first givenfrequency. The given exposure time is generally selected such thatimages acquired by a camera 218 are not blurry when the device is movingat a given speed. For example, at speeds used by mobile automationapparatuses within retail environments (e.g. about 0.3 m/sec), eachcamera 218 is operated at about 5 Hz, with an exposure time of 0.5 ms.

However, other frequencies and exposure times are within the scope ofpresent embodiments. When the apparatus 103 is to move at a speed slowerthan about 0.3 m/s, the frequency is decreased while exposure time isincreased, and when the apparatus 103 is to move at a speed faster thanabout 0.3 m/s, the frequency is increased while the exposure time isdecreased. Regardless, the given exposure time is generally selectedsuch that images acquired by a camera 218 are not blurry when the deviceis moving at a given speed. The frequency is selected such thatsuccessively acquired images by a given camera 218 at least partiallyoverlap at the given speed.

However, operating a paired light source 219 at the same first frequencyand for a pulse duration that corresponds to the given exposure time canlead to strobing and/or flickering of the paired light source 219 thatmay be visible to an observer, for example either to a sensitive humanor animal or to an electronic imaging device operating based on humanvision system parameters.

An example, of an electronic imaging device operating based on humanvision system parameters includes a light or imaging sensor that senseslight using an image recognition rate that models human ability toprocess images.

While the light sources 219 may be operated in an always on manner toeliminate visibility of their successive activations, the light sources219 are generally operated at about 500 W to ensure that features inimages acquired by the cameras 218, including images of bar codes, aredetectable and/or decodable. As the apparatus 103 is powered from abattery, each light source 219 being always on would significantlyshorten the battery life.

Hence, to obviate such strobing, and to reduce the effect on batterylife, for a camera 218 (configured to acquire images periodicallyaccording to a given exposure time, and at a first given frequency), atleast one paired light source 219 is configured to periodically provideillumination light for the camera 218, the illumination light having apulse duration that corresponds to the given exposure time and having asecond given frequency that is an integer multiple of the first givenfrequency, wherein the second given frequency is higher than a thresholdfrequency where successive activations of the at least one light sourceare imperceptible, for example to the observers discussed above. Putanother way, the second given frequency is higher than a thresholdfrequency where operation of the light source 219 appears to becontinuous and/or always on, for example to the above observers. Put yetanother way, above the threshold frequency, the strobing and/orflickering of the light source 219 is imperceptible.

In some example embodiments, the threshold frequency is greater than 70Hz. Hence, when a camera 218 is operated at a frequency of 5 Hz, with anexposure time of 0.5 ms, a paired light source 219 is operated at afrequency that is an integer multiple of 5 Hz that is above 70 Hz, andwith a pulse duration of 0.5 ms that is synchronized with the exposuretime of the camera 218.

Operating cameras 218 and light sources 219 at such frequencies, andwith such short exposure times (e.g. on the order of 0.5 ms) and/orpulse durations can be challenging. As such, in particular embodiments,the cameras 218 each comprise a “fast” CMOS (complementarymetal-oxide-semiconductor) device, and the like, while each of the lightsources 219, in an example embodiment, comprise at least one lightemitting diode (LED) capable of providing around 500 W of illumination.However, other types of cameras and light sources are within the scopeof the present specification. For example, a CCD (charge couple device)may be used in place of and/or in addition to, the CMOS device,presuming the CCD device is compatible with the exposure times.

Furthermore, a paired light source 219 for a paired camera 218 islocated at a distance from the paired camera 218 that illuminates anobject imaged by the paired camera 218, and where parasitic reflectionsfrom the paired light source 219 are not reflected into the pairedcamera 218. For example, such parasitic reflections include glare fromshiny surfaces of the products 112 and/or labels 201 and/or shiny and/orreflective surfaces in front of each, such as from the reflective door203. Furthermore, a given paired light source 219 and paired camera 218are operated at a different time from other paired light sources 219 andother paired cameras 218 located where the parasitic reflections fromthe given paired light source 219 are reflected into the other pairedcameras 218.

Referring now to FIG. 3, a schematic block diagram of further componentsof the apparatus 103 is depicted. In particular, the apparatus 103includes a special purpose controller 320 specifically designed tocontrol the cameras 218 and the light sources 219, as described herein.The controller 320 is interconnected with a non-transitory computerreadable storage medium, such as a memory 322. The memory 322 includesany suitable combination of volatile (e.g. Random Access Memory or RAM)and non-volatile memory (e.g. read only memory or ROM, ElectricallyErasable Programmable Read Only Memory or EEPROM, flash memory). Ingeneral, the controller 320 and the memory 322 each comprise one or moreintegrated circuits. The controller 320, for example, includes anysuitable combination of central processing units (CPUs) and graphicsprocessing units (GPUs) and/or any suitable combination offield-programmable gate arrays (FPGAs) and/or application-specificintegrated circuits (ASICs) configured to implement the specificfunctionality of the apparatus 103. Further, as depicted, the memory 322includes a repository 332 for storing data, for example data collectedby sensors 330. In an embodiment, the controller 320, further includesone or more central processing units (CPUs) and/or graphics processingunits (GPUs). In an embodiment, a specially designed integrated circuit,such as a Field Programmable Gate Array (FPGA), is designed to controlthe cameras 218 and the light sources 219 as discussed herein, eitheralternatively or in addition to the controller 320 and memory 322. Asthose of skill in the art will realize, the mobile automation apparatus103 can also include one or more speicial purpose controllers orprocessors and/or FPGAs, in communication with the controller 320,specifically configured to control navigational and/or data captureaspects of the mobile automation apparatus 103.

The apparatus 103 also includes a communications interface 324interconnected with the controller 320. The communications interface 324includes any suitable hardware (e.g. transmitters, receivers, networkinterface controllers and the like) allowing the apparatus 103 tocommunicate with other devices—particularly the server 101 and,optionally, the mobile device 105—for example via the links 107, and thelike. The specific components of the communications interface 324 areselected based on the type of network or other links over which theapparatus 103 is required to communicate, including, but not limited to,the wireless local-area network of the retail environment of the system100.

The memory 322 stores a plurality of applications, each including aplurality of computer readable instructions executable by the controller320. The execution of the above-mentioned instructions by the controller320 configures the apparatus 103 to perform various actions discussedherein. The applications stored in the memory 322 include a controlapplication 328, which may also be implemented as a suite of logicallydistinct applications. In general, via execution of the controlapplication 328 or subcomponents thereof, the controller 320 isconfigured to implement various functionality. As will now be apparent,some or all of the functionality implemented by the controller 320described below may also be performed by preconfigured hardware elements(e.g. one or more ASICs) rather than by execution of the controlapplication 328 by the controller 320.

The apparatus 103 further includes sensors 330, including, but notlimited to, LiDAR (Light Detection and Ranging) sensors, as well asother navigation sensors and/or data acquisition sensors (other than thecameras 218 and the light sources 219) and/or obstacle avoidancesensors.

The apparatus 103 further includes a navigation module 340 configured tomove the apparatus 103 in an environment, for example the environment ofthe modules 110. The navigation module 340 comprises any suitablecombination of motors, electric motors, stepper motors, and the likeconfigured to drive and/or steer the wheels 212, and the like, of theapparatus 103.

Hence, in general, the controller 320 is configured to control theapparatus 103 to navigate the environment of the module 110 using datafrom the navigation sensors to control the navigation module 340including, but not limited to, avoiding obstacles. For example, whilenot depicted, the memory 322 further stores a map, and the like, of theenvironment, which is used for navigation in conjunction with thenavigation sensors.

The controller 320 is in communication with the cameras 218 and thelight source 219 which are powered by a battery 399. Indeed, theapparatus 103 is generally powered by the battery 399.

In the present example, in particular, the apparatus 103 is configuredvia the execution of the control application 328 by the controller 320,to control the cameras 218 and light sources 219 to acquire images (e.g.digital images, and the like) used for one or more of stock detection,low stock detection, price label verification, plug detection, planogramcompliance and the like.

Turning now to FIG. 4, before describing the operation of theapplication 328 to control the cameras 218 and light sources 219 toacquire images, certain components of the application 328 will bedescribed in greater detail. As will be apparent to those skilled in theart, in other examples the components of the application 328 may beseparated into distinct applications, or combined into other sets ofcomponents. Alternatively or in addition, some or all of the componentsillustrated in FIG. 4 may also be implemented as dedicated hardwarecomponents (e.g. one or more special purpose FPGAs and/or one or moreASICs connected to the controller 320).

The control application 328 includes a frequency coordinator 401. Inbrief, the frequency coordinator 401 is configured to coordinate thefrequency of operation of a pair of a camera 218 and a light source 219providing illumination light for the camera 218.

The frequency coordinator 401 includes: a camera frequency controller418 configured to control a frequency of operation of a camera 218; anda light source frequency controller 419, configured to control afrequency of operation of a light source 219 paired with the camera 218being controlled by the camera frequency controller 418.

The control application 328 further includes an image acquisitionsequencer 451. In brief, the frequency coordinator 401 is configured tosequence image acquisition of the plurality of cameras 218 inconjunction with controlling the plurality of light sources 219.

The image acquisition sequencer 451 includes: a camera sequencer 458configured to sequence operation of the cameras 218; and a light sourcesequencer 459 configured to sequence operation of the light sources 219in conjunction with paired with the camera being controlled by thecamera frequency controller 418.

The frequency coordinator 401 and the image acquisition sequencer 451are used in conjunction with each other to control and coordinaterespective frequencies of operation of a light source 219 with a pairedcamera 218, and to sequence operation of the plurality of cameras 218with their paired light sources 219.

Attention is now directed to FIG. 5 which depicts a flowchartrepresentative of an example method 500 for coordinating coordinaterespective frequencies of operation of a light source 219 with a pairedcamera 218. The example operations of the method 500 of FIG. 5correspond to machine readable instructions that are executed by, forexample, the apparatus 103, and specifically by the controller 320and/or the various components of the control application 328 including,but not limited to, the frequency coordinator 401. Indeed, the examplemethod 500 of FIG. 5 is one way in which the apparatus 103 isconfigured. However, the following discussion of the example method 500of FIG. 5 will lead to a further understanding of the apparatus 103, andits various components. However, it is to be understood that in otherembodiments, the apparatus 103 and/or the method 500 are varied, andhence need not work exactly as discussed herein in conjunction with eachother, and that such variations are within the scope of presentembodiments.

Furthermore, the example method 500 of FIG. 5 need not be performed inthe exact sequence as shown and likewise, in other embodiments, variousblocks may be performed in parallel rather than in sequence.Accordingly, the elements of method 500 are referred to herein as“blocks” rather than “steps.” The example method 500 of FIG. 5 may beimplemented on variations of the example apparatus 103, as well.

At block 501, the controller 320 controls a camera 218 to acquire imagesperiodically according to a given exposure time, and at a first givenfrequency.

At the block 503, the controller 320 controls at least one light source219 (e.g. paired with the camera controlled at the block 501) toperiodically provide illumination light for the camera 218, theillumination light having a pulse duration that corresponds to the givenexposure time and having a second given frequency that is an integermultiple of the first given frequency, wherein the second givenfrequency is higher than a threshold frequency where successiveactivations of the at least one light source are imperceptible.

The method 500 is now described with reference to FIG. 6, whichschematically depicts a timing diagram 600 of frequency of operation ofa camera 218, and concurrent frequency of operation of a paired lightsource 219. The timing diagram 600 is understood to be schematic and theratio of the camera 218 on times to off times is not to scale, andindeed neither is the ratio of the light source 219 on time to off timesto scale.

In particular, a pulse train 618 shows exposure pulses of a camera 218that is controlled (e.g. at the block 501 and/or using the camerafrequency controller 418) to periodically operate according to a firstfrequency and according to a given exposure time (e.g. 0.5 ms). In otherwords, presuming that the first given frequency is “N” Hz, every 1/Nseconds, the camera 218 is controlled to expose the CMOS device, and thelike, for the given exposure time. Hence, if the given exposure time is0.5 ms, and the first frequency is 5 Hz, every 200 ms, the camera 218 isexposed for 0.5 ms.

Furthermore, a pulse train 619 shows “on” pulses of a light source 219operated to provide illumination light for the camera 218 operatingaccording to a pulse train 618. The light source 219 is controlled (e.g.at the block 503 and/or using the light source frequency controller 419)to periodically provide illumination light for the camera 218. Theillumination light has a pulse duration that corresponds to the givenexposure time of the camera (e.g. 0.5 ms) and has a second givenfrequency that is an integer multiple of the first given frequency,wherein the second given frequency is higher than a threshold frequencywhere successive activations of the at least one light source areimperceptible.

Furthermore, the light pulses are coordinated with the exposure of thecamera 218. For example, presuming that “M” is an integer, the secondgiven frequency is M*N Hz, and hence the light source 219 provides alight pulse every 1/(M*N) seconds. Furthermore, in some exampleembodiments, as depicted, every M^(th) light pulse being coincident withan exposure of the camera 218 such that every M^(th) light pulseprovides illumination for the camera 218 while the camera 218 isacquiring an image. Put another way, every M^(th) light pulse iscoincident in time with a corresponding camera exposure.

For example, if there are 20 light pulses (e.g. “M” light pulses) foreach camera exposure, the second given frequency of operation of thelight source 219 would be 100 Hz.

Furthermore, if the given pulse duration time is 0.5 ms (e.g. the sameas the camera exposure time), and the second given frequency is 100 Hz(there are 20 light pulses for each camera exposure and/or “M”=20),every 10 ms, the light source 219 provides a light pulse that is 0.5 mslong. The light source 219 is hence only “on” 5% of the time, whichreduces the impact of the light source 219 on the life of the battery399. Furthermore, as the frequency of operation of the light source 219is above a threshold frequency where successive activations of the lightsource 219 are imperceptible, the chance of the light source 219inducing discomfort to an observer is eliminated.

The threshold frequency generally depends on the pulse duration of thelight source 219. For example, for pulse durations around 0.5 ms, thethreshold frequency has been determined to be above 70 Hz for manyobservers. However, as the pulse duration increases (e.g. along with theexposure time of a paired camera 218) the threshold frequency maydecrease for some observers; and similarly, as the pulse durationdecreases (e.g. along with the exposure time of a paired camera 218) thethreshold frequency may increase for some observers.

Those of skill in the art will realize that other combinations of cameraexposure times, light pulse duration times, and frequencies are withinthe scope of present embodiments. For example, the camera exposure timeand camera operation frequency is generally selected based on the speedof the apparatus 103 to ensure that successive images acquired by samecamera 218 at least partially overlap. As the speed of the apparatus 103increases or decreases, the camera exposure time and camera operationfrequency are adjusted accordingly, and hence so is the light pulseduration and the light source frequency.

Either way, the light source 219 is operated at a frequency higher thanthe threshold frequency where successive activations of the light source219 are imperceptible to an observer making the light source 219 appearto be continuous and/or always on.

It is further appreciated that each of the plurality of camera 218,along with their respective paired light sources 219, are controlledaccording to the method 500. However, as light sources 219 that are notpaired with a given camera 218 can cause glare, and/or other types ofparasitic reflections, the operation of the cameras 218 and paired lightsources 219 are generally sequenced.

Hence, attention is now directed to FIG. 7 which depicts a flowchartrepresentative of an example method 700 for sequencing image acquisitionusing cameras and light sources. The example operations of the method700 of FIG. 7 correspond to machine readable instructions that areexecuted by, for example, the apparatus 103, and specifically by thecontroller 320 and/or the various components of the control application328 including, but not limited to, the image acquisition sequencer 451.Indeed, the example method 700 of FIG. 7 is one way in which theapparatus 103 is configured. However, the following discussion of theexample method 700 of FIG. 7 will lead to a further understanding of theapparatus 103, and its various components. However, it is to beunderstood that in other embodiments, the apparatus 103 and/or themethod 700 are varied, and hence need not work exactly as discussedherein in conjunction with each other, and that such variations arewithin the scope of present embodiments.

Furthermore, the example method 700 of FIG. 7 need not be performed inthe exact sequence as shown and likewise, in other embodiments, variousblocks may be performed in parallel rather than in sequence.Accordingly, the elements of method 700 are referred to herein as“blocks” rather than “steps.” The example method 700 of FIG. 7 may beimplemented on variations of the example apparatus 103, as well.

At block 701, the controller 320 controls a paired light source 219 anda paired camera 218 to operate at a different time from other pairedlight sources 219 and other paired cameras 218 located where theparasitic reflections from the paired light source 219 are reflectedinto the other paired cameras 218.

At block 703, the controller 320 controls the paired light source 219and the paired camera 218 (i.e. of the block 701) to operate at a sametime as associated paired light sources 219 and associated pairedcameras 218, the paired camera 218 (i.e. of the block 701) and theassociated paired cameras 218 each located where respective parasiticreflections from the paired light source 219 (i.e. of the block 701) andthe associated paired light sources 219 are not reflected into thepaired camera 218 and the associated paired cameras 218.

Indeed, the blocks 701, 703 are generally executed concurrently todetermine which pairs of cameras 218 and light sources 219 are to beoperated at any given time using, for example, the camera sequencer 458and the light source sequencer 459. Furthermore, the method 700 isgenerally dependent on a geometry and/or physical arrangement of thecameras 218 and light sources 219.

Hence, attention is next directed to FIG. 8, FIG. 9, and FIG. 10 whichdepict geometry of the cameras 218 and the light sources 219 relative toeach other and a module 110. Indeed, each of FIG. 8, FIG. 9, and FIG. 10are similar to a portion of FIG. 2, with like elements having likenumbers.

In FIG. 8 only one light source 219-1 and one camera 218-1 are indicatedfor clarity. However, FIG. 8 depicts fields of view 818 of the camera218, depicted as dashed lines, which at least partially overlap, forexample, at least in plane of their respective focal distance and/or ina plane of their respective focal depths, each of which are generallyselected to about coincide with the items being imaged, for example theretail products 112 and/or the labels 201. Indeed, while the cameras 218are generally assumed herein to have similar focal distances and/orsimilar depths of field, in other embodiments, at least one of thecameras 218 has a different focal distance and/or a different depth offield from the other cameras 218. Furthermore, it is generally assumedthat the apparatus 103 is navigated relative to the module 110 such thatthe items to be imaged are within the respective focal distances and/orin the respective focal depths of the cameras 218. As adjacent fields ofview 818 overlap, the entire vertical length of the module 110, at leastfrom the top camera 218 to the bottom camera 218 is imageable.

Attention is next directed to FIG. 9 which depicts a given camera 218-2being operated to image the product 112 behind the glass door 203, asindicated by its field of view 818. In FIG. 9, an adjacent light source219-4, is operated to provide illumination light 988 to illuminate theproduct 112 behind the glass door 203. However, the light source 219-4is located such that parasitic reflections 989 from the illuminationlight 988, are reflected into the camera 218-2, for example by the door203, however such parasitic reflections 989 can also be caused by asurface of the product 112 being shiny, and the like.

Hence, as the light source 219-4 and the camera 218-2 are locatedrelative to each other such that the parasitic reflections 989 from thelight source 219-4 enter the camera 218-2, the light source 219-4 is notpaired with the camera 218-2. Indeed, the situation depicted in FIG. 9represents an issue that is at least partially addressed by the method700.

Attention is next directed to FIG. 10 which depicts the given camera218-2 being operated to image the product 112 behind the glass door 203,as indicated by its field of view 818. In FIG. 10, the adjacent lightsource 219-4 is not operated due the parasitic reflections 989 asdescribed with regard to FIG. 9. However, a paired light source 219-5(e.g. the next light source down from the adjacent light source 219-4)is paired with the camera 218-2, as the paired light source 219-5 islocated at a distance 1099 from the paired camera 218-2 that bothilluminates an object (e.g. the product 112 behind the glass door 203)imaged by the paired camera 218-2, and where parasitic reflections fromthe paired light source 219-5 are not reflected into the paired camera218-2. As is apparent from FIG. 10, at least one camera 218-3 is locatedbetween the paired light source 219-5 and the paired camera 2189-2 alongthe mast 214.

For example, as depicted, the paired light source 219-5 emitsillumination light 1088 that illuminates the product 112 behind the door203, such that light 1090 from the product 112, that at least partiallycomprises the illumination light 1088 interacting with the product 112imaged by the paired camera 218-2. However, the light 1090 does notinclude parasitic reflections. As depicted, however, parasiticreflections 1089 from the paired light source 219-5 are reflected intoanother camera 218-1, adjacent the paired camera 218-2; as such, thecamera 218-1 (and any light sources 219 paired with the camera 218-1)are not operated during operation of the paired light source 219-5 andits paired camera 218-2.

Put another way, the controller 320 (e.g. at the block 701 of the method700) controls the paired light source 219-5 and its paired camera 218-2to operate at a different time from other paired light sources 219 andother paired cameras 218 located where the parasitic reflections fromthe paired light source 219-5 are reflected into the other pairedcameras 218.

It is further appreciated that the paired light source 219-5 and itspaired camera 218-2 are also operated according to the method 500, asdescribed above.

It is yet further appreciated that other cameras 218 located whereparasitic reflections from the paired light source 219-5 are notreflected therein can be operated at the same time as the paired lightsource 219-5 and its paired camera 218-2, as long as the other lightsources 219 do not produce parasitic reflections for any of the cameras218 being operated.

Put another way, the controller 320 (e.g. at the block 703 of the method700) controls the paired light source 219-5 and its paired camera 218-2to operate at a same time as associated paired light sources 219 andassociated paired cameras 218, the paired camera 218-2 and theassociated paired cameras 218 each located where respective parasiticreflections from the paired light source 219-5 and the associated pairedlight sources 219 are not reflected into the paired camera 218-2 and theassociated paired cameras 218.

Put yet another way, the plurality of cameras 218 are sequenced tooperate with their respective paired light sources 219 such that anygiven camera 218 is not receiving parasitic reflections and to maximizethe number of cameras 218 that are simultaneously acquiring images.

Sequencing of the cameras 218 with paired light sources 219 is nowdescribed with respect to FIG. 11 which schematically depicts aperspective view of the cameras 218 and their paired light sources 219,as well as associations therebetween. While not depicted, it is assumedthat the cameras 218 and the light sources 219 are arranged along themast 214 of the apparatus 103.

Referring to FIG. 11, it is apparent that each light source 219 islocated in rows along the mast 214 and/or support structure,perpendicular to a line between the plurality of cameras 218. Forexample, as depicted, each light source 219 comprises a plurality oflight emitting diodes (and the like), indicated by a respective arrow.Furthermore, a direction of a respective arrow indicates a pitch of arespective LED and/or light source. For example, in some embodiments,LEDs (and the like) indicated by an “up” arrow are pitched upwards at anangle of about 30°, while LEDs (and the like) indicated by a “down”arrow are pitched downwards at an angle of about 30°. However, otherangles are within the scope of present embodiments. In particular, apitch of a light source 219 is selected to provide illumination lightfor a paired camera 218 and such that parasitic reflections do not occurfor the paired camera 218. In other words, in example embodiments, eachof the plurality of light sources 219 is pitched, with respect to themast 214 and/or support structure, in a direction of an associatedpaired camera 218.

From FIG. 11 it is further apparent that the cameras 218 and the lightsources 219 are organized into three channels, each channel operated insequence, with all cameras 218 and all light sources 219 in a givenchannel being associated and operated simultaneously, with the cameras218 and the light sources 219 in the other channels operated at adifferent time.

Hence, in a given channel, none of the light sources 219 produceparasitic reflections for the cameras 218 in the given channel.

The channel with which each of the cameras 218 and the light sources 219are associated is indicated both by a number adjacent each of thecameras 218 and the light sources 219 (e.g. “1”, “2”, or “3”,indicating, respectively, association with channel 1, channel 2, orchannel 3), as well as graphically, with sets of cameras 218 associatedwith a respective channel being graphically the same, and set of lightsources 219 associated with a respective channel indicated by the samerespective arrow type.

Hence, for example, the cameras 218-1, 218-4, 218-7 and the lightsources 219-1, 219-4, 219-7, 219-10 are in channel 1, the cameras 218-2,218-5, 218-8 and the light sources 219-2, 219-5, 219-8 are in channel 2,and the cameras 218-3, 218-6, and the light sources 219-3, 219-6, 219-9are in channel 3.

Furthermore, a given camera 218 (e.g. in a given channel) is paired withat least a first subset of the plurality of light sources 219 located onone side of the given camera 218, and a second subset of the pluralityof light sources 219 located on an opposite side of the given camera218. For example, camera 218-1, of channel 1, is paired with the LEDs(and the like) of the light source 219-1 of channel 1, and the camera218-1 is paired with the LEDs (and the like) of the light source 219-4(also of channel 1) which are pitched towards the camera 218-1 (e.g.indicated by the direction of the arrows).

Furthermore, the camera 218-4 of channel 1 is paired with the LEDs (andthe like) of the light source 219-4 (also of channel 1) that are pitchedtowards the camera 218-4 (e.g. as indicated the direction of thearrows), as well as the LEDs (and the like) of the light source 219-7(also of channel 1) that are pitched towards the camera 218-4.

Hence, the LEDs (and the like) in each row of a light source 219 aregenerally operated at a same time, as each of the LEDs in a row of alight source 219 is associated with paired cameras 218 in a givenchannel. In other words, the geometry of the LEDs in each row is suchthat for all cameras 218 operated while all the LEDs in a row (as wellfor other light sources) are operated, parasitic reflections do notoccur. Put another way, in depicted embodiments, associated subsets ofthe plurality of light sources 219 are located in rows along the mast214, perpendicular to a line between the plurality of cameras 218, andat least some of the rows of the associated subsets of the plurality oflight sources 219 are operated at a same time to provide illuminationlight for at least two of the plurality of cameras 218. Some rows,however, provide illumination light for only one of the plurality ofcameras 218.

In any event, from FIG. 11 it is hence apparent that for any lightsource 219, some of the LEDs (and the like) can be paired with onecamera 218 in a channel, while other LEDs (and the like) can be pairedanother camera 218 in the same channel. However, as long as the variousLEDs, and the like, are in the same channel, they are operatedsimultaneously.

From FIG. 11, some of the light sources 219 include four LEDs (and thelike) and others include two LEDs (and the like). For the light sources219 which include four LEDs (and the like), two of the LEDs are pitchedin one direction (e.g. the two outer LEDs), while the two other LEDs arepitched in an opposite direction (e.g. the two inner LEDs). However, thearrangement and pitch of the LEDs, and the like, for each light source219 is generally determined heuristically, given a particular geometryof the mast 214 and/or support structure onto which the cameras 218 andlight sources 219 are being provisioned.

Indeed, as depicted, the outer LEDs are 250 mm apart (center-to-center),the cameras 218 are 250 mm apart (center-to-center), and a camera 218 toan adjacent light source row is 125 mm (center-to-center). Furthermore,the two top light sources 219-1, 219-2 include only two LEDs (and thelike) each, and are paired, respectively, with camera 218-1 and camera218-2.

Indeed, in some embodiments, at least a portion of the cameras 218 arepaired with light sources 219 located on either side.

Operation of the channels is described with reference to FIG. 12 whichdepicts a timing diagram 1200 that depicts a portion of three pairs ofpulse trains, one for each of channel 1, channel 2 and channel 3,showing exposure times and pulse for each camera 218 and light source219 in each of the channels. Each of the pairs of the pulse trains arethe same as the pulse trains 618, 619 of FIG. 6, but offset from oneanother such that during any given exposure time for a set of cameras218 of a given pulse train, the light sources 219 of the other channelsare off. Each pair of pulse trains otherwise operate according to themethod 500. In other words, for clarity only a portion of each of thepair of pulse trains are depicted showing only a single camera exposureevent for each channel.

Hence, from FIG. 12, it is apparent that, for each channel, paired lightsources and paired cameras are operated at a different time (e.g. at theblock 701 of the method 700) from other paired light sources and otherpaired cameras (e.g. at the other channels); it is further apparentthat, for each channel, paired light sources and paired cameras areoperated at a same time (e.g. at the block 703 of the method 700) asassociated paired light sources and associated paired cameras (e.g. ofthe same channel).

Indeed, the plurality of cameras 218 are operated in a sequence withassociated paired light sources 219, for example, using a channel-basedarchitecture. Indeed, once the geometry of the cameras 218 and the lightsources 219 are provisioned, and the cameras 218 and the light sources219 are grouped into channels, according to the geometry of theparasitic reflections produced by the light sources 219, the cameras 218and the light sources 219 are operated according to a timing scheme, forexample as represented by FIG. 12.

While not depicted, once the images from the cameras 218 are acquired,the images are transmitted to the server 101 for processing.

While the cameras and light sources have been described herein as beinglocated along a common housing and/or support structure, in otherembodiments the cameras and light sources described herein are locatedin respective housings and/or respective support structures andpositioned relative to one another using any suitable components.

In any event, provided herein is a device, and in particular a mobileautomation apparatus, that includes cameras and light sources configuredto acquire clear images while moving, including, but not limited todecodable images of barcodes using, for example, a short exposure timeon the cameras. During this short exposure time, a large amount ofillumination is generally used to ensure that details of the images aredecodable, for example on the order of 500 W.

However, the high brightness of the light sources causes potentialissues with glare off of products, label covers, and glass doors(freezer section of grocery dept.). Hence, the cameras and the lightsources are physically separated such that the chances of parasiticreflection from the light sources to the camera (e.g. glare) is reduced.However, as a plurality of cameras are used, light sources adjacent to agiven camera that are used to provide illumination for a camera that isfurther away, can cause glare on the adjacent camera. Hence, the camerasand light sources are operated in a sequence such that no camera isacquiring images while an adjacent and/or glare producing light sourceis on. Furthermore, the frequency and pulse duration of the lightsources is controlled to reduce the impact on a battery as well as toeliminate potential for discomfort to an observer.

In the foregoing specification, specific embodiments have beendescribed. However, one of ordinary skill in the art appreciates thatvarious modifications and changes can be made without departing from thescope of the invention as set forth in the claims below. Accordingly,the specification and figures are to be regarded in an illustrativerather than a restrictive sense, and all such modifications are intendedto be included within the scope of present teachings.

The benefits, advantages, solutions to problems, and any element(s) thatmay cause any benefit, advantage, or solution to occur or become morepronounced are not to be construed as a critical, required, or essentialfeatures or elements of any or all the claims. The invention is definedsolely by the appended claims including any amendments made during thependency of this application and all equivalents of those claims asissued.

Moreover, in this document, language of “at least one of X, Y, and Z”and “one or more of X, Y and Z” can be construed as X only, Y only, Zonly, or any combination of two or more items X, Y, and Z (e.g., XYZ,XY, YZ, XZ, and the like). Similar logic can be applied for two or moreitems in any occurrence of “at least one . . . ” and “one or more . . .” language.

Moreover, in this document, relational terms such as first and second,top and bottom, and the like may be used solely to distinguish oneentity or action from another entity or action without necessarilyrequiring or implying any actual such relationship or order between suchentities or actions. The terms “comprises,” “comprising,” “has”,“having,” “includes”, “including,” “contains”, “containing” or any othervariation thereof, are intended to cover a non-exclusive inclusion, suchthat a process, method, article, or apparatus that comprises, has,includes, contains a list of elements does not include only thoseelements but may include other elements not expressly listed or inherentto such process, method, article, or apparatus. An element proceeded by“comprises . . . a”, “has . . . a”, “includes . . . a”, “contains . . .a” does not, without more constraints, preclude the existence ofadditional identical elements in the process, method, article, orapparatus that comprises, has, includes, contains the element. The terms“a” and “an” are defined as one or more unless explicitly statedotherwise herein. The terms “substantially”, “essentially”,“approximately”, “about” or any other version thereof, are defined asbeing close to as understood by one of ordinary skill in the art, and inone non-limiting embodiment the term is defined to be within 10%, inanother embodiment within 5%, in another embodiment within 1% and inanother embodiment within 0.5%. The term “coupled” as used herein isdefined as connected, although not necessarily directly and notnecessarily mechanically. A device or structure that is “configured” ina certain way is configured in at least that way, but may also beconfigured in ways that are not listed.

It will be appreciated that some embodiments may be comprised of one ormore generic or specialized processors (or “processing devices”) such asmicroprocessors, digital signal processors, customized processors andfield programmable gate arrays (FPGAs) and unique stored programinstructions (including both software and firmware) that control the oneor more processors to implement, in conjunction with certainnon-processor circuits, some, most, or all of the functions of themethod and/or apparatus described herein. Alternatively, some or allfunctions could be implemented by a state machine that has no storedprogram instructions, or in one or more application specific integratedcircuits (ASICs), in which each function or some combinations of certainof the functions are implemented as custom logic. Of course, acombination of the two approaches could be used.

Moreover, an embodiment can be implemented as a computer-readablestorage medium having computer readable code stored thereon forprogramming a computer (e.g., comprising a processor) to perform amethod as described and claimed herein. Examples of suchcomputer-readable storage mediums include, but are not limited to, ahard disk, a CD-ROM, an optical storage device, a magnetic storagedevice, a ROM (Read Only Memory), a PROM (Programmable Read OnlyMemory), an EPROM (Erasable Programmable Read Only Memory), an EEPROM(Electrically Erasable Programmable Read Only Memory) and a Flashmemory. Further, it is expected that one of ordinary skill,notwithstanding possibly significant effort and many design choicesmotivated by, for example, available time, current technology, andeconomic considerations, when guided by the concepts and principlesdisclosed herein will be readily capable of generating such softwareinstructions and programs and ICs with minimal experimentation.

The Abstract of the Disclosure is provided to allow the reader toquickly ascertain the nature of the technical disclosure. It issubmitted with the understanding that it will not be used to interpretor limit the scope or meaning of the claims. In addition, in theforegoing Detailed Description, it can be seen that various features aregrouped together in various embodiments for the purpose of streamliningthe disclosure. This method of disclosure is not to be interpreted asreflecting an intention that the claimed embodiments require morefeatures than are expressly recited in each claim. Rather, as thefollowing claims reflect, inventive subject matter lies in less than allfeatures of a single disclosed embodiment. Thus, the following claimsare hereby incorporated into the Detailed Description, with each claimstanding on its own as a separately claimed subject matter.

We claim:
 1. A device comprising: a camera, configured to acquire imagesperiodically according to a given exposure time, and at a first givenfrequency; and at least one light source configured to periodicallyprovide illumination light for the camera, the illumination light havinga pulse duration that corresponds to the given exposure time and havinga second given frequency that is an integer multiple of the first givenfrequency, wherein the second given frequency is higher than a thresholdfrequency where successive activations of the at least one light sourceare imperceptible.
 2. The device of claim 1, wherein each of the givenexposure time and the pulse duration are about 0.5 ms.
 3. The device ofclaim 1, wherein the first given frequency is about 5 Hz and the secondgiven frequency is about 100 Hz.
 4. The device of claim 1, wherein thethreshold frequency is greater than 70 Hz.
 5. The device of claim 1,wherein the at least one light source comprises at least one lightemitting diode.
 6. The device of claim 1, wherein the second givenfrequency is higher than the threshold frequency where the successiveactivations of the at least one light source are imperceptible to ahuman vision system.
 7. The device of claim 1, wherein the at least onelight source and the camera are powered by a battery.
 8. The device ofclaim 1, wherein the given exposure time is selected such that theimages acquired by the camera are not blurry when the device is movingat a given speed.
 9. A method comprising: at a device comprising acontroller, a camera and at least one light source, controlling, usingthe controller, the camera to acquire images periodically according to agiven exposure time, and at a first given frequency; and controlling,using the controller, the at least one light source to periodicallyprovide illumination light for the camera, the illumination light havinga pulse duration that corresponds to the given exposure time and havinga second given frequency that is an integer multiple of the first givenfrequency, wherein the second given frequency is higher than a thresholdfrequency where successive activations of the at least one light sourceare imperceptible.
 10. The method of claim 9, wherein the second givenfrequency is higher than the threshold frequency where the successiveactivations of the at least one light source are imperceptible to ahuman vision system.
 11. A device comprising: a support structure; aplurality of cameras spaced along the support structure, whereinadjacent cameras have partially overlapping fields of view directed awayfrom the support structure; and, a plurality of light sources comprisingat least one light source paired with each of the plurality of cameras,wherein a paired light source is located at a distance from a pairedcamera that illuminates an object imaged by the paired camera, and whereparasitic reflections from the paired light source are not reflectedinto the paired camera, and wherein the paired light source and thepaired camera are operated at a different time from other paired lightsources and other paired cameras located where the parasitic reflectionsfrom the paired light source are reflected into the other pairedcameras.
 12. The device of claim 11, wherein the paired light source andthe paired camera are operated at a same time as associated paired lightsources and associated paired cameras, the paired camera and theassociated paired cameras each located where respective parasiticreflections from the paired light source and the associated paired lightsources are not reflected into the paired camera and the associatedpaired cameras.
 13. The device of claim 11, wherein the plurality ofcameras is operated in a sequence with associated paired light sources.14. The device of claim 11, further comprising a mobile automationapparatus, the support structure disposed on the mobile automationapparatus, the plurality of cameras and the plurality of light sourcesspaced along the support structure.
 15. The device of claim 11, furthercomprising a mobile automation apparatus, the support structurecomprising a mast disposed on the mobile automation apparatus, theplurality of cameras and the plurality of light sources verticallyspaced along the mast.
 16. The device of claim 11, wherein at least onecamera is located between the paired light source and the paired cameraalong the support structure.
 17. The device of claim 11, wherein each ofthe plurality of light sources is pitched, with respect to the supportstructure, in a direction of an associated paired camera.
 18. The deviceof claim 11, wherein a given camera, of the plurality of cameras ispaired with at least a first subset of the plurality of light sourceslocated on one side of the given camera, and a second subset of theplurality of light sources located on an opposite side of the givencamera.
 19. The device of claim 11, wherein associated subsets of theplurality of light sources are located in rows along the supportstructure, perpendicular to a line between the plurality of cameras, andwherein at least some of the rows are operated at a same time to provideillumination light for at least two of the plurality of cameras.
 20. Amethod comprising: at device comprising: a controller; a supportstructure; a plurality of cameras spaced along the support structure,wherein adjacent cameras have partially overlapping fields of viewdirected away from the support structure; and, a plurality of lightsources comprising at least one light source paired with each of theplurality of cameras, wherein a paired light source is located at adistance from a paired camera that illuminates an object imaged by thepaired camera, and where parasitic reflections from the paired lightsource are not reflected into the paired camera, controlling, using thecontroller, the paired light source and the paired camera to operate ata different time from other paired light sources and other pairedcameras located where the parasitic reflections from the paired lightsource are reflected into the other paired cameras; and controlling,using the controller, the paired light source and the paired camera tooperate at a same time as associated paired light sources and associatedpaired cameras, the paired camera and the associated paired cameras eachlocated where respective parasitic reflections from the paired lightsource and the associated paired light sources are not reflected intothe paired camera and the associated paired cameras.